Method for texturizing virtual three-dimensional objects

ABSTRACT

The invention relates to a method for texturizing virtual three-dimensional objects, particularly virtual three-dimensional building objects and city models with a photographic image ( 1 ) of a real object, particularly of a picture of a façade. The method is characterized by the following steps: Projecting the photographic image ( 1 ) onto a virtual surface ( 2 ) of the virtual three-dimensional object to produce a raw texture; localizing a raw texture element ( 3 ) in the raw texture by using a classification method; computer-compatible description of the localized raw texture element by a formal feature set for the raw texture element, particularly a feature vector; comparing the formal feature set of the raw texture element with each feature set of predefined library elements ( 4 ), and determining degrees of similarity between the raw texture element and each library element; replacing the localized raw texture element with at least one library element when a predefined degree of similarity is present, and reshaping the raw texture into a generalized texture ( 5 ) of the virtual object by replacing all raw texture elements with library elements.

The invention relates to a method for texturing virtualthree-dimensional objects, particularly virtual three-dimensionalbuilding objects and city models, according to the preamble of claim 1.

With respect to a graphical representation of a virtualthree-dimensional object generated by a computer, the texture ortexturing, respectively, is the image displayed on the surface of thethree-dimensional object. In a digital textured 3D modeling used for theplanning of buildings or cities, for example, façade surfaces aredisplayed on the surface of the objects, which represent views of thefaçades of real objects, i.e. real houses or other buildings, with acertain recognition value.

For texturings of this type, two methods are currently available. In afirst method, a photorealistic texturing is performed. A photographicimage of the surface of the real object is projected onto the surface ofthe virtual object. As a rule, picture editing methods are used for thispurpose. In such a method, many details of the real object arereproduced due to the use of real image data, so that the virtualthree-dimensional object allows a high informative content.

However, this method requires a high image quality of the photographicsource material, which may not be guaranteed from the beginning and,depending on the image material, may be difficult to correctsubsequently. In general, such a texturing also requires a high degreeof manual processing work, during which the textures of the virtualobjects are checked for correctness one by one and the real photographicimage data have to be adapted to the virtual object, especially scaledand, if necessary, freed from faults. Due to the use of real image datathe generated virtual objects require a great data volume, and even ifthe representation of the object is scaled down, all image informationof the original image material have to be stored together with thevirtual object.

As an alternative, also a generic texturing may be applied. Thegraphical elements are positioned on the surface of the virtual objectin the form of a schematic representation of the real object, so thatthe entire surface of the virtual model is generated artificially. Sucha representation reproduces fewer details of the real object. The soachieved informative content is much smaller than that of aphotorealistic representation. It is, however, an advantage of such amethod that already prefabricated graphical elements are used accordingto the modular design principle, so that a high degree of automation canbe achieved. Also, the generated virtual objects are significantlysmaller as far as their data volumes are concerned as compared to thephotorealistically textured 3D objects.

In the practical use of virtual 3D objects, where it should, on the onehand, be possible to reproduce and process the three-dimensional modelwith as many details as possible while keeping the storage capacity low,it shows very frequently that there is no practicable method to ensure agreat richness of details for the 3D objects on the one hand, and a lowdata volume on the other hand. In many cases, the optimum reproductionof a real object in the virtual model is exactly at a point somewherebetween the photorealistic and the generic representation, a virtualobject need not reproduce all details of the real object, but the purelygeneric representation of the object is too rough and short of details.

Also, both texturing methods require in many cases a very considerablemanual expenditure. Depending on the complexity of the object in thespecific case, the texturing may constitute up to 60% of the overallexpenditure of a project. In typical projects, where three-dimensionalcity models are generated on the computer, currently—despite highlyeffective individual solutions in some cases—a great amount of work hasto be accomplished manually, so that thousands of man hours of highlyspecialized technical personnel may be necessary.

Therefore, it is the object to provide a method, by means of which thetime expenditure needed for the texturing can be reduced significantlyby an extensive automation, so that the cost efficiency and, last butnot least, also the competitiveness can be improved considerably.Specifically, the operator is to be relieved from the time-consuming andwearisome activity entailed by transforming the object views into thetexture of the three-dimensional objects.

Moreover, it is the object to make the use of oblique aerialphotographs, which are obtained as air photographs, available tolarge-scale projects by automated procedures, and to combine theeffective acquisition of real object surfaces and real textures with apossibly fast and effective transformation into textures for virtualthree-dimensional objects.

Finally, it is the object to provide a method for texturing virtualthree-dimensional objects, in which the texture of the virtual objectcorresponds with sufficient exactness to the photorealisticrepresentation, while the storage capacity of the virtual object isreduced to a maximum and approximately corresponds to the generictexturing. Specifically, the texturing method is to guarantee that thecharacteristic properties of the represented objects, which are visiblein the photorealistic representation, are reproduced on the virtualobject to a greatest possible extent or as completely as possible,although the virtual object has a substantially generic surface.Specifically, the method is to allow that the picture of the real objectcan be transferred into the generic representation as individually aspossible and with a maximum of automation.

The objects are achieved with a method for texturing virtualthree-dimensional objects, particularly virtual three-dimensionalbuilding objects and city objects, with a photographic image of a realobject, particularly a picture of a façade. The method is characterizedby the following inventive method steps:

In a first step, the photographic image is projected onto thecorresponding surface of the virtual three-dimensional object togenerate a raw texture.

Within the raw texture, a raw texture element is localized by using aclassification method. The localized raw texture element is described ina computerized manner by a formal attribute set, particularly anattribute vector.

The formal attribute set of the, raw texture element is compared with anattribute set of predetermined library elements, and similarity measuresbetween the raw texture element and a library element are determined.

The localized raw texture element is now replaced by a library element,which has a similarity measure of a defined size and/or quantity. Inconnection therewith, finally the raw texture is transformed into ageneralized texture of the virtual object by replacing all raw textureelements by library elements.

Thus, the method according to the invention is based on the idea tocollect at first possibly all structure-forming elements on aphotorealistic texture, i.e. the raw texture elements whichsubstantially define the appearance of the texture. This includes, forexample, the shape and color of the façade and the plaster, especiallydecorative elements on the façade, windows and rows of windows, doors,balconies and the like shapes and structures, which are recognizable ina photorealistic image of the real object. Thus, this step representsthe detection and localization of the respective raw texture element.

This localization is combined with a classification method, whereby thecharacteristics of the raw texture element, e.g. color, shape, size,location and position are captured in the photograph image and arestored.

The so detected characteristics form a formal set of attributes or anattribute vector, respectively, by means of which a raw texture elementcan uniquely and comparably be defined, detected and edited for asubsequent data processing.

This formal set of attributes is now compared with attribute sets ofpredetermined library elements. It may also be used for the constructionof library elements.

The library elements are picture components or picture elements, fromwhich a generalized texture can be assembled. The comparison between theformal attribute set and the attribute set of the library element isintended to find a suited library element by which the raw textureelement can be replaced. To this end, a similarity measure between theformal attribute set of the raw texture element and the attribute set ofthe library element is determined.

Provided that the similarity measure corresponds to a predefined value,e.g. a maximum, a threshold or a defined tolerance range, the rawtexture element is now replaced by the respective library element. Now,the library element appears in the texture at the place of the previousraw texture element.

By replacing additional raw texture elements by corresponding libraryelements, thus, a generalized texture is generated from the totaloriginal raw texture. This means that the generalized texture is builtup on the basis of the identified details of the raw texture. As aresult of the comparison between the raw texture element and the libraryelement it shows a greatest possible similarity to the raw texture, witha great and defined degree of detail. However, due to the generic basicstructure its data volume is minimal. The capturing of the raw textureelements, the attribute classification thereof and the comparison oftheir attributes with the attributes of the library elements isformalized and automated, the insertion of the library elements isaccomplished by image processing steps, which are likewise automated.Thus, the generation of the generalized texture is largely automatic.

The photographic image for generating the raw texture may derive fromvarious sources. Particularly advantageous is the use of a georeferencedterrestrial digital photograph, where the location and the direction ofthe shot can be uniquely identified in the terrestrial coordinatesystem. Also, such a photograph is already available in a digital form.

Another advantageous source for the projected photographic image is anair photograph, particularly a nadir or an oblique aerial photograph.

Expediently, image processing steps are carried out to improve the imageby removing faults in the raw texture, specifically a reduction and/orelimination of disadvantageous shadow edges and a deblurring and/orimproving the definition. Thus, it is avoided that artefacts, whichexclusively result, for example, from exposure conditions during theshooting of the image data, are treated as raw texture elements in thefurther course of the procedure.

The classification method for localizing the raw texture elementincludes a detection of position, shape, color, surfaces and/or edgesaccording to previously defined search parameters, whereby the localizedraw texture element is selected at least in view of its position in theraw texture, its shape, color and/or edge structure and the likeattributes.

Such a procedure entails, for example, that doors or windows on a façadeare detected and are identified to be different from other façadeelements, such as stucco strips or downpipes, due to their rectangularshape. Moreover, structures such as window crosses and skylights areadditionally identified. The so detected object is identified, forexample, as a window, especially as a window with a skylight or windowcross, respectively.

Usefully, scalable attributes of the raw texture element, particularlythe height and width and/or picture element numbers of the raw textureelement, are rescaled to a normalized reference quantity. This referencequantity then forms a part of the formal attribute set of the rawtexture element. Such an approach reduces the library elements whichmight be necessary for the exchange of a raw texture element. Thus, itis particularly possible to provide substantially only one libraryelement for a number of windows that have an invariable height/widthratio, even when being rescaled. The normalized reference quantity nowmakes it possible to perform a very simple comparison between the rawtexture element and the library element and to test the correspondenceof both elements in an uncomplicated manner.

Scaling-independent attributes of the raw texture element, particularlycolor values, represent a predominantly absolute reference quantity inthe formal attribute set of the raw texture element. However, a colorscaling for differently illuminated parts of the façades is possible,for example, to represent effects of light and shadows. One example forthis is a façade color tone which does not experience any change even ifthe size of the façade is scaled. Such an attribute can only adopt anabsolute value and, therefore, can be compared in absolute terms only.

The comparison of the formal attribute set of the raw texture elementwith the attribute set of a library element includes a comparison of thenormalized reference quantities, with a similarity test being carriedout between a first normalized reference quantity and a secondnormalized reference quantity.

In this test it is proceeded from the fact that ratios of sizes remaininvariant in similarity representations. Accordingly, a raw textureelement and a library element are similar if this test entails apositive result. The library element can, in this case, be transferredinto the raw texture element by a similarity transformation, therebyoverlapping with its shape the image area of the raw texture element.

The comparison of the formal attribute set of the raw texture elementwith the attribute set of the library element further includes acomparison of the absolute reference quantities, whereby a test for agreatest possible correspondence of the absolute reference quantities iscarried out.

This test substantially detects the degree of correspondence betweenabsolute quantities, e.g. color values. It is a direct comparisonbetween characteristics of the raw texture element and the libraryelement, wherein an adaptation of the next best available variant of thelibrary element to the raw texture element is, in principle, possible bycorrespondingly varying the respective absolute quantity of the libraryelement.

In the determination of the similarity measure between the formalattribute set of the raw texture element and the attribute set of thelibrary element a degree of correspondence of the absolute referencequantities and/or a stability of the invariant ratios is determined. Thegreatest similarity measure between the raw texture element and thelibrary element is obtained if the absolute reference quantitiescorrespond to each other sufficiently well in an optionally narrowtolerance range and the ratios between the normalized referencequantities are as stable as possible.

In an expedient embodiment the similarity measure is defined in advance.All library elements are here outputted as selection alternatives with asimilarity measure lying within a tolerance range to act as possiblereplacements for the raw texture element, sorted according to thesimilarity value.

The replacement of the raw texture element by the library element isaccomplished by cutting the point set of the raw texture element out ofthe raw texture and inserting the point set of the library element intothe raw texture. Basically, a cut-out/insert method known as cut andcopy is carried out, where as many areas of the raw texture as possibleare replaced by library elements so as to transfer the raw texture intothe generalized texture as completely as possible.

This replacement procedure expediently includes a manual postprocessing,allowing possibly required corrections to be made.

In practice, this is accomplished by inserting sections, that are notclassified in the raw texture, into the generalized texture, expedientlyas pixel groups, specifically bitmaps. These non-classified areas, e.g.ornaments having a complicated shape, then represent an image componentsubsequently inserted into the generalized texture.

To achieve an economical use of process capacities, especially ofstorage resources and computation time, an at least partial tiling ofthe virtual object with a periodic sequence of a library element adaptedto a raw texture element can be performed when the raw texture istransformed into the generalized texture, whereby library elements aresubstantially adapted only for one location of the raw texture, withthese adapted elements being joined in a tile-like manner and coveringthe virtual object.

The method according to the invention shall be explained in more detailby means of an embodiment in connection with FIGS. 1 to 8. Likereference numbers shall be used for like or equally acting method stepsand components.

In the figures:

FIG. 1 shows an exemplary terrestrial photograph of a house façade in ablack and white coding,

FIG. 2 shows a virtual three-dimensional object generated for the housefaçade of FIG. 1, in a view corresponding to the photograph of FIG. 1,

FIG. 3 shows basic method steps in a general representation,

FIG. 4 shows exemplary raw texture elements on the previously shownfaçade picture,

FIG. 5 shows scalings and comparisons of attribute sets between the rawtexture element and the library element by the example of a window and awindow cross,

FIG. 6 shows an exemplary tree structure for some library elements,

FIG. 7 shows replacements of raw texture elements by library elements bythe example of some window shapes,

FIG. 8 shows replacements of non-classified raw texture structures bythe example of stucco elements of a doorway arch.

FIG. 1 shows a façade picture 1, and FIG. 2 shows the pertinent surfaceof the correspondingly generated virtual three-dimensional object 2 in aperspective corresponding to the picture of FIG. 1. The façade pictureshown in FIG. 1 has been generated in the form of a terrestrial digitalphotograph. Expediently, the terrestrial image is georeferenced. Thismeans that the location of the photographic device in the coordinatesystem of the earth and the orientation thereof with respect to thecoordinate axes at the shooting location is uniquely known.

Instead of the terrestrial photograph, also oblique or nadir aerialphotographs can be used. The virtual three-dimensional object shown inFIG. 2 is a contour abstracted from the real object, i.e. the façade orthe pertinent house, the boundaries of which reproduce the bodyboundaries of the real object at a reduced scale with sufficientexactness. Specifically, the virtual contour must be exact enough toallow the insertion of the captured façade picture on the correspondingrim of the virtual object true to size and free of distortion.

The comparison of the representations shown in FIG. 1 and FIG. 2 showsthat building shapes and contours, such as dormers or also continuousfaçade lines, may be used as references and reference points for theadaptation of the façade picture to the rim of the virtualthree-dimensional object.

FIG. 3 shows basic method steps of the method according to theinvention. In a method step a, the façade picture 1 is applied astexture onto the virtual three-dimensional object, onto a surfaceprovided therefor. This texture will be referred to as raw texturebelow. To generate the raw texture, software modules already providedfor this purpose may by used, which are normally used for generatingphotorealistic textures. Thus, the generation of the raw texture issubstantially accomplished fully automatically.

Immediately after the generation of the raw texture, or already prior tothe generation thereof, an image processing may be carried out on theset of the picture elements of the façade picture so as to improve theimage quality and prepare the structure recognitions. This concernsabove all shadow shapes and disturbing edge structures caused by thesame on the image data. To this end, above all a deblurring of the edgesmay be performed, or their definition may be improved.

It is particularly advantageous and, as a rule, necessary for asubsequent identification of raw texture elements to transform anoriginally colored façade picture into a façade picture in the form of acopy in a gray tone or black and white mode. After such a colortransformation, windows and window crosses stand out against a lighterfaçade background by very distinct black surfaces intersected by lightlines, which can be seen in the figures. Such a transformation is alsoadvantageous if the structure of façade designs with different colors isto be identified. In the example of FIG. 3, particularly thestrip-shaped clinker elements are well identifiable as a result oftransforming the image into the black and white mode.

However, during a reduction to a black and white image representationmany information get lost because a great amount of different colorvalues are mapped to only a few b/w values, whereby the thresholds forthe allocation between the color range and the black and white range arechosen more or less arbitrarily. This problem can be avoided by using aclassificator capable of analyzing colors. It allows the detectabilityof the raw texture elements in respect of their contour, with anallocation of library elements if different colors are concerned.

In a step b, the classification method is applied to the so generatedraw texture, whereby special raw texture elements 3, e.g. windowstructures or other façade elements such as balconies, doors, stuccoetc., are localized on the raw texture on the one hand, and are comparedwith previously stored library elements 4 on the other hand. In theexample shown, the raw texture elements are formed as windows, whichhave a typical rectangular shape with a certain ratio between height andwidth and typical shapes of window crosses, and which moreover show atypical color or contrast effect. As library elements 4 image elementsare available, to which the captured raw texture elements are comparedin respect of their attributes. The localization of the raw textureelements, the description thereof and the comparison thereof with thelibrary elements forms the classification method step. The conclusionthereof is the selection of a library element which reproduces thecharacteristics of the raw texture element with a particularly greatsimilarity. A detailed explanation of the classification shall be givenbelow.

By using the library elements, the entire raw texture is transformed toa generalized texture 5 in a final step c, whereby all classified rawtexture elements, i.e. windows and other structures on the façade, arereplaced by library elements and the generic texture is assembledaccording to the modular design principle. In the example shown in FIG.3, all windows from the raw structure and the clinker structuresextending over the façade as vertical strips are replaced by thecorresponding library elements.

The classification of exemplary raw texture elements shall be explainedin more detail below.

FIG. 4 shows some exemplary raw texture elements, especially windows andstucco elements. The exemplary façade picture comprises two differentdormer windows 31 and 32. In this example, dormer window 31 consists oftwo single windows located closely adjacent to each other. Dormer window32 is a single window with a T-shaped window cross. Window 31 and window32 both stand clearly out against the background with their whiteframes.

In this example, windows 33, 34 and 35 substantially have black shapes,broken by a white T-shaped window cross, which clearly stand out againstthe substantially white background of the façade. The shape of window 36corresponds substantially to that of windows 33 to 35, but is influencedby an object located behind the panes, which affects the black panesurfaces and makes them appear inhomogeneous. This effect is moreclearly shown in another window 36, where a curtain affects the blackstructure of the pane and nearly renders the design of the window crossirrecognizable.

Finally, stucco elements 37, 38 and 39 are provided, which may beconsidered as additional raw texture elements and the contour of whichis indistinct against the background of the façade.

The raw texture elements 31 and 32 or 33 to 35, respectively, can belocalized on the black-and-white-coded picture by a shape recognition.To this end, the picture elements of the raw texture are evaluated,wherein the areas of the black panes standing out against the whitesurroundings and the shape of the window crosses are read out by asoftware. Above all, the rectangular shape of the black window contoursand the regular arrangement thereof with respect to each other areidentified. In this image recognition, the white window crosses formwhite sets of picture elements arranged in strips of a well-definedwidth and length, which are inserted between the rectangular blacksurfaces of the window panes. The picture element set formed of whitestrips and black surfaces, together with the position thereof in thepicture, is then identified as a raw texture element and read out.

To search for and identify windows, particularly window distributionsassumed a priori are used on the façade surface. Thus, if the façadesurface has a height of 15 m, it is assumed that the representedbuilding has four floors each with a height of 3.75 m. Based on thesestarting parameters a search for four rows of windows at a correspondingdistance is proposed and possibly executed. It shows that such aproposal automatism shows surprisingly good results.

Inhomogeneities such as in connection with the window structure 36 canusually be removed or suppressed by an image processing, which precedesthe localization of the raw texture elements. To this end, on principleseveral image processings are possible. Specifically, inhomogeneitiescan be removed by color-coding the façade picture in an expedientmanner. Regular structures such as window frames and window crosses thusstand out due to their uniform coloring and are then easier to identifyfor the structure recognition. The black and white coding shown in thefigure is particularly advantageous if regular structures are to belocalized on a light façade. As a rule, when processing the images insuch a way, the correspondingly recoded image is then used as a copy forthe structure recognition.

The stucco elements 37 and 39 are characterized by irregular andcomplicated structures, which clearly stand out against the unstructuredand white façade, however. Such areas can be read out either as surfaceswith a certain average gray tone, or these image areas can be cut out inthe form of a bitmap. Corresponding method steps shall be explained inmore detail below.

FIG. 5 shows an example of the classification method used for a windowcross. The figure shows a raw texture element obtained by the imagerecognition on the left. In order to classify the raw texture element,the set of picture elements determined in the image recognition has tobe compared with a library element 4. To this end, a structurecomparison, a size comparison and a color comparison are substantiallyperformed, for which purpose attribute sets between the raw textureelement and the library element are compared.

The size of the raw texture element plays a significant role in theattribute set thereof. Of importance is, in this case, not primarily theabsolute size of the raw texture element, however, because it varieswith each scaling action and can, therefore, not be compared with anabsolute size of a library element. It is rather checked when comparingthe size of the raw texture element with that of the library elementwhether the dimensions comply with the geometric laws for similaritytransformations.

FIG. 5 shows, for example, a raw texture element 3 with a width b and aheight h₁. Moreover, the raw texture element comprises a window cross,the transverse beam of which is located at a height h₂, while thevertical beam of the window cross divides the width of the raw textureelement at a foot ratio b₁/b₂. It may be assumed that a raw textureelement and an optional library element correspond to each other withrespect to their shapes if the ratio of sizes of optional sections inthe raw texture element and the library element, respectively, coincide.On this condition, a library element can be scaled to the shape of theraw texture element by means of the similarity transformation.

Advantageously, ratios of sizes for an attribute set of the raw textureelement are now determined. In the example of FIG. 5, above all, theratio of sizes between width and height b/h₁=v₁, the ratio between theheight of the transverse beam and the total height h₂/h₁=v₂ and the footratio b₁/b₂=v₃ offer themselves.

Moreover, color information of individual image areas of the raw textureelement are read out from the image data of the raw texture. In theexample shown, especially the colors f₁ and f₂ of the window surfacesand the window cross or the color f₃ of the window frame aresignificant. The exemplary attribute set of the raw texture element canthen be represented as a formal attribute vector, which completelydescribes the raw texture element.

m=(v₁; v₂; v₃; f_(l); f₂; f₃)

The values of the attribute vector m remain unchanged even if the rawtexture element is scaled. In the raw texture element 3′, for example,the ratios v₁ to v₃ and f₁ to f₃ remain constant, although thedimensions of the raw texture element now being smaller have adoptedother values.

FIG. 5 shows in contrast thereto a library element 4 to which, inanalogy to the characteristic values defined in relation to the rawtexture element, a comparable attribute vector is allocated from thevery beginning. The ratio of width and height B/H₁=V₁, the ratio of theheight of the transverse beam to the total height H₂/H₁=V₂ and the footratio B₁/B₂=V₃ of the library element as well as the color values F₁,F₂, F₃ of the color areas of the library element are defined in exactlythe same manner and are combined to an attribute vector M of the libraryelement:

M=(V₁; V₂; V₃; F₁; F₂; F₃)

For the classification, the attribute vector m of the raw textureelement is compared to the attribute vector of the library elementcomponent by component, whereby a similarity measure between bothvectors is determined. The tolerances for the deviations of theindividual values in the comparison may be predefined and basically bevaried as desired. Advantageously, the tolerance ranges for thedeviations of the ratios v_(n) and V_(n) in both vectors are chosen tobe tighter and the deviation tolerances for the deviations of the colorvalues are chosen to be greater. A correct reproduction of the size orthe ratios of sizes of the raw texture element by the library elementis, accordingly, more important than a correct color reproduction. Ofcourse, such preferences can be selected differently according to therequirement.

As to the color values f_(n) and F_(n), the standard color systems forimage processing programs, specifically the RGB or CMYK systems, may beused. In the example shown in FIG. 5, three library elements areavailable for selection, each of which have color values F_(1a), F_(2a),F_(3a); F_(1b), F_(2b), F_(3b), and F_(1c), F_(2c), F_(3c). The finallychosen library element corresponds, in the example shown, to the libraryelement with the colors F_(1b), F_(2b), F_(3b). Thus, the raw textureelement is classified with respect to its color as well as its size.

FIG. 6 shows an exemplary and strongly simplified tree structure for themanagement of library elements. The library elements in this embodimentare primarily organized according to functional criteria. For example,library elements for windows, doors, façade surfaces etc. are contained.Basically, it is also possible to organize library elements according toother criteria, particularly purely graphical ones. In this case, thetree would include a branch for rectangular, circular, elliptical andthe like other library elements, regardless of their functionalreferences. Such an alternative tree structure would insofar be moreadvantageous as the raw texture elements are localized by the imageprocessing mainly by means of the shape.

Moreover, also two tree structures may be provided as a combination,with the library elements being simultaneously managed in both ways.

The classification of the library elements according to their functiondoes have certain advantages, however. Specifically, a user canpredefine in advance that raw texture elements, which are capturedwithin a specific predetermined image section in the raw texture, aretreated exclusively as a window or exclusively as a door or the likeelement, respectively. In a way, this allows a faster classification ofthe raw texture elements because, for example, a raw texture elementhaving a T-shaped image structure, which is not in the proximity of thelower image edge, refers to a window. In this case, in order to find thecorrect library element, a search algorithm would not search among allrectangular shapes, but would switch to the directory “windows” rightaway.

In the example shown herein there are provided a directory A_(l) forhigh rectangular windows, a directory A₂ for wide rectangular windowsand a directory A₃ for round or differently shaped windows. Thedirectory A₂ likewise includes additional subgroups A₂₁ and A₂₂, whichare related to different divisions of the window surface.

The exact relationships between height and width of the library elementsaccording to the description in connection with FIG. 5 need not be fixedfrom the beginning, however. They may be changed at any time byvariations, by stretching or shrinking the height and width. The logicaldifferentiation between high and wide rectangular windows is, however,sensible in so far as, according to experience, certain window crossshapes are only used for certain window types.

Thus, the directory A₁ includes a subgroup A₁₁ for windows with askylight, a subgroup A₁₂ for bipartite windows with a skylight, and asubgroup A₁₃ for bipartite windows with a bipartite skylight. Thelibrary elements of the respective subgroup are linked with attributevectors which are characterized by different sets of parameters. Thus,for example, the attribute vectors of the subgroup A₁₁ do not comprise aparameter for the above-mentioned foot ratio because no window cross isprovided.

The mathematical configuration of the attribute vector, i.e. the numberof the parameters and components provided, accordingly represents thegraphical construction of the library element. Expediently, the databasefor the library elements contains primarily data vectors and datafields, which are dimensioned in correspondence with the attributevectors, but substantially no graphic representations or images of thelibrary elements themselves. Graphically, the library elements aregenerated by corresponding program routines only when the correspondingpartial areas of the generalized texture are generated.

Thus, the volume of the database can be kept very compact. Moreover,predefining the structure of the attribute vectors and some boundaryconditions permits, for example, to represent the aforementioneddirectories of type A_(l) and A₂ by one single configuration of theattribute vector, whereby the components of this attribute vector arevaried in correspondence with the library elements contained in thedirectories or are assigned corresponding values, respectively.

It will be appreciated that according to the description of FIG. 5, anumber of color models for the respective library elements are stored inany of the subdirectories, with an attribute vector with a correspondingset of parameters according to the preceding description being assignedto each individual library element. Moreover, also different libraryelements with different height/width ratios can be predefined in advanceas fixed models with fixed characteristics. These fixed models withdefined parameters in the attribute vectors may be used in a very roughand simple procedure for the standardized replacement of raw textureelements.

The directory T comprises library elements for doors. As per expedientdefinition, doors are all raw texture elements which are located in thearea of a lower image edge or a lower edge of the three-dimensionalvirtual object or the raw texture, respectively. Exemplarysubdirectories are formed by a directory T₁ for single doors and adirectory T₂ for double doors.

Moreover, a directory for façade surfaces is provided. This directorysubstantially comprises fill patterns or fill images with differentcolors and structures, which are dimensioned according to a patternrecognition executed on the raw texture. Thus, for example, there isprovided a directory P₁ for rough plaster and a directory P₂ for aclinker façade. The library elements stored in these directories aresubstantially bitmaps or other graphics files in which patterns arestored via defined brightness and color values, which are compared togiven structures in the raw texture. In the simplest case, the area Pcontains a simple color palette with different color values, which arecompared to colors on the raw texture.

The different subdirectories and the different library elements storedin the same, respectively, may also be combined. Thus, for example, acolor surface of a library element A “window” can be filled with a coloror a pattern from the subdirectory P “pattern”, whereby a first routinedetermines according to the above description the shape and type of thelibrary element “window”, while a second routine determines andclassifies the exact color or pattern structures from the raw texture.Therefore, it should be pointed out in this connection that theindividual subdirectories and sections in the tree diagram of FIG. 6 maybe assigned different classification routines.

Another domain of the library elements, which is not shown herein, isdefined by simple decorative elements for façades, e.g. cornices,keystones, simple stucco elements etc., whose assignment tocorresponding raw texture elements is substantially accomplished by acombination of shape recognition and scaling, as was already describedin connection with the example shown in FIG. 5.

The replacement of raw texture elements by the library elements isaccomplished by cutting them out and replacing them. This means that thearea of the classified raw texture element in the raw texture is cut outand replaced by the library element. Here, it may be necessary that thelibrary element has to be scaled in respect of its size.

FIG. 7 shows some exemplary replacements of raw texture elements bylibrary elements by the example of windows with façade decorations. Theexamples shown in FIG. 7 are to illustrate above all how the generated;generalized texture 5 is assembled from modules. The figure shows theraw textures on the left side and the generalized texture generatedtherefor on the right side. Example a) shows a window with a triangularstucco element 40 a, a rectangular section 41 a underneath the same anda window lintel 42 a. Moreover, columnar flanks 43 a are provided. Thewindow surface 44 a is formed of a window cross with a divided skylight.

In this example, the stucco element 40 a in the generalized texture 5 isreplaced by a triangle 40 b, which is partly filled with a darker colorso as to imitate the shadow structure of the stucco element 40 a.Section 41 a is replaced by a rectangle 41 b enclosed by darker sectionson the left and right. The window lintel 42 a and the flanks 43 b arealso replaced by rectangles. According to the preceding description, acorresponding library element is used for the window surface 44 b. Theelements 40 b, 41 b, 42 b, 43 b and 44 b now define the generalizedtexture 5 of the raw texture element in question, that is, of the windowshown on the left.

In example b), the raw texture element comprises flanks 43 a of asimpler design, and a window surface 44 a only showing a transversebeam. In addition to these simple attributes, however, a decorativeelement 45 a is provided, which shows a relatively complicated picturemotif, in this example a lion's head.

This lion's head represents a non-classifiable element. This means thateither its structure cannot be detected in an image recognition, or thatno library element exists for its structure. In this case, the imagearea of the lion's head is transferred as bitmap 45 b into thegeneralized texture illustrated on the right. Together with the libraryelements 43 b and 44 b the bitmap 45 b forms the correspondinggeneralized texture in example b) of FIG. 7. This generalized texturesimultaneously forms a mixture of one section of the raw texture elementand multiple replacements by individual library elements. Thus, the rawtexture element can be transferred into the generalized texture withsufficient exactness, with the attribute of the lion's head beingpreserved.

Example c) shows a raw texture element in the form of a window with anundecorated frame and a wedge-shaped keystone 47 a in connection with aclinker façade 46 a. In the generalized texture on the right thekeystone is formed by a library element 47 b in the form of a trapezoid.The clinker façade was classified in a pattern recognition and isreplaced'by a library element 46 b with a corresponding fill structure“wall”.

Specific combinations of individual library elements are combined andstored in special visualization libraries to allow a fastvisualization/use of the textured models. A visualization library ishere usable for one or more projects.

FIG. 8 shows a final example for a treatment of non-classifiablestructures in the raw texture. In this example, a doorway arch is shown,which comprises a lion's head as a first raw texture 48 a, variousornaments 49 a and 50 a. The comparison with the generalized texture 5shows that, in this example, the raw textures 48 a and 49 a weretransferred into the generalized texture as bitmaps, while the rawtexture 50 a is a picture section provided with a uniform color fillingand now forms a component 50 b of the generalized texture. Such atreatment of non-classifiable raw texture structures is above all usefulif extensive areas with memory-intensive picture element structures arenot classifiable and if the transfer of these structures as bitmapswould therefore result in relatively complicated generalized textures.In this case, for example, a mean color or brightness value of the areain question can be determined and used as filling for the respectivearea of the generalized texture.

Instead of replacing each single façade element by an elementary libraryelement, the method can be simplified significantly by tiling thevirtual object. This always offers itself if highly periodic façadestructures have to be reproduced, which is specifically the case withpanelized building structures and apartment houses.

One reason for this approach results from the formal operating principleof standard visualization software and from the necessity to increasethe visualization speed of the textured virtual object, because thetextured model objects to be displayed in real-time have to be assembledin a dynamic process from the vector data of the building body and theaddressed elements of the supplied library during the visualization.

Instead of the replacement of individual elementary library elementsdescribed above, a tile structure is generated from repeated libraryelements, which covers the total surface of the textured object. Thisparticularly means that instead of the individual attributes, such as awindow of type A and a façade color of type B, an adapted wall elementis generated, which contains combined attributes of types A/B. By acyclic repetition this combined element completely fills the texturedsurface. This procedure has proved to be very effective.

Of course, with all generated generalized textures, or during thegeneration of the generalized textures, respectively, a “manual” imagepostprocessing is possible, i.e. by using an image processing program,so as to compensate certain inaccuracies. To this end, the generalizedtexture is outputted in the form of a known image data format, wherebyvectorized data formats allowing scalings are particularly expedient.Especially scalable SVG vector graphics have proved to be very suitablefor this purpose.

The method was explained by means of embodiments. The person skilled inthe art will appreciate that modifications to the illustratedembodiments may be made without departing from the fundamental idea ofthe invention. Other modifications and embodiments are particularlydefined in the dependent claims.

LIST OF REFERENCE NUMBERS

-   1 façade picture-   2 virtual three-dimensional object-   3 raw texture element-   4 library element-   5 generalized texture-   31 raw texture element first dormer window-   32 raw texture element second dormer window-   33 raw texture element first window-   34 raw texture element second window-   35 raw texture element third window-   36 raw texture element with inhomogeneous structure-   37 raw texture element, stucco-   38 raw texture element, stucco-   39 raw texture element, stucco-   40 a raw texture element, stucco element, triangular-   40 b library element, triangle filled with color-   41 a raw texture element, rectangular window lintel-   41 b library element, rectangular shape-   42 a raw texture element, window lintel-   42 b library element, rectangle-   43 a raw texture element, flank-   43 b library element, rectangle-   44 a raw texture element, window surface with window cross-   44 b library element, window surface with cross-   45 a raw texture, lion's head-   45 b transferred bitmap, lion's head in generalized texture-   46 a raw texture, clinker façade-   46 b library element, fill structure wall-   48 a raw texture lion-   48 b generalized texture, inserted bitmap-   49 a raw texture first ornament-   49 b generalized texture, inserted bitmap-   50 a raw texture second ornament-   50 b generalized texture, homogeneous color filling-   b raw texture element, width-   h₁ raw texture element, height-   h₂ raw texture element, height transverse beam window cross-   v₁ raw texture element, width/height ratio-   v₂ raw texture element, transverse beam/total height ratio-   v₃ raw texture element, foot ratio-   f_(n) raw texture element, color value different areas-   m raw texture element, formal attribute vector-   B library element, width-   H₁ library element, height-   H₂ library element, height transverse beam window cross-   V₁ library element, width/height ratio-   V₂ library element, transverse beam/total height ratio-   V₃ library element, foot ratio-   F_(n) library element, color value different sections-   M attribute vector, library element-   A₁ directory high rectangular windows-   A₁₁ windows with skylight-   A₁₂ windows, bipartite with skylight-   A₁₃ windows, bipartite, bipartite skylight-   A₂ directory broad rectangular windows-   A₃ directory round windows-   T directory doors-   T₁ single doors-   T₂ double doors-   P₁ structure directory rough plaster-   P₂ structure directory clinker façade

1. Method for texturing virtual three-dimensional objects, particularlyvirtual three-dimensional building objects and city models with aphotographic image (1) of a real object, particularly a façade picture,characterized by the following method steps: projecting the photographicimage (1) onto a virtual surface (2) of the virtual three-dimensionalobject to generate a raw texture, localizing a raw texture element (3)within the raw texture by using a classification method, describing thelocalized raw texture element in a computerized manner by a formalattribute set for the raw texture element, particularly an attributevector, comparing the formal attribute set of the raw texture elementwith an attribute set of predetermined library elements (4) anddetermining similarity measures between the raw texture element and alibrary element, replacing the localized raw texture element by at leastone library element if a predefined similarity measure exists,transforming the raw texture into a generalized texture (5) of thevirtual object by replacing all raw texture elements by libraryelements.
 2. Method according to claim 1, characterized in that theprojected photographic image for generating the raw texture is obtainedfrom a georeferenced terrestrial digital photograph.
 3. Method accordingto claim 1, characterized in that the projected photographic image isobtained from an air photograph, particularly a nadir or an obliqueaerial photograph.
 4. Method according to claim 1, characterized in thatimage processing steps are carried out to improve the image by removingfaults in the raw texture, specifically a reduction and/or eliminationof disadvantageous shadow edges and blurrings by means of suitedmethods.
 5. Method according to claim 1, characterized in that theclassification method for localizing the raw texture element includes adetection of position, shape, color, surface and/or edge according topreviously defined search parameters, whereby the localized raw textureelement is selected at least in view of its position, shape, color,surface and/or edge structure and the like attributes.
 6. Methodaccording to claim 1, characterized in that if the attributes of the rawtexture element, particularly a height and width and/or a pictureelement number, are scaling-dependent, the raw texture element isresealed to a normalized reference quantity and the normalized referencequantity forms a part of the formal attribute set of the raw textureelement.
 7. Method according to claim 1, characterized in thatscaling-independent attributes of the raw texture element, particularlycolor values, define an absolute reference quantity in the formalattribute set of the raw texture element.
 8. Method according to claim1, characterized in that the comparison of the formal attribute set ofthe raw texture element with the attribute set of a library elementincludes a comparison of the normalized reference quantities, with asimilarity test being carried out between a first normalized referencequantity and a second normalized reference quantity if the ratio isinvariant when being scaled.
 9. Method according to claim 1,characterized in that the comparison of the formal attribute set of theraw texture element with the attribute set of a library element includesa comparison of the absolute reference quantities, whereby a test for agreatest possible correspondence of the absolute reference quantities iscarried out.
 10. Method according to claim 1, characterized in that inthe determination of the similarity measure a degree of correspondenceof the absolute reference quantities and/or a stability of the invariantratio is evaluated.
 11. Method according to claim 1, characterized inthat the similarity measure is defined in advance, with all libraryelements being outputted as selection alternatives for the raw textureelement with a similarity measure lying within a tolerance range. 12.Method according to claim 1, characterized in that the replacement ofthe raw texture element by the library element is accomplished bycutting the point set of the raw texture element out of the raw textureand inserting the point set of the at least one library element into theraw texture.
 13. Method according to claim 12, characterized in that thereplacement includes a manual postprocessing step.
 14. Method accordingto claim 1, characterized in that non-classified sections of the rawtexture are inserted into the generalized texture as pixel groups,particularly bitmaps.
 15. Method according to claim 1, characterized inthat during the transformation of the raw texture into the generalizedtexture a tiling of the virtual object is carried out at least partiallyby a repeated insertion of an adapted library element.
 16. Methodaccording to claim 2, characterized in that image processing steps arecarried out to improve the image by removing faults in the raw texture,specifically a reduction and/or elimination of disadvantageous shadowedges and blurrings by means of suited methods.
 17. Method according toclaim 3, characterized in that image processing steps are carried out toimprove the image by removing faults in the raw texture, specifically areduction and/or elimination of disadvantageous shadow edges andblurrings by means of suited methods.
 18. Method according to claim 5,characterized in that if the attributes of the raw texture element,particularly a height and width and/or a picture element number, arescaling-dependent, the raw texture element is rescaled to a normalizedreference quantity and the normalized reference quantity forms a part ofthe formal attribute set of the raw texture element.
 19. Methodaccording to claim 5, characterized in that scaling-independentattributes of the raw texture element, particularly color values, definean absolute reference quantity in the formal attribute set of the rawtexture element.
 20. Method according to claim 2, characterized in thatthe comparison of the formal attribute set of the raw texture elementwith the attribute set of a library element includes a comparison of thenormalized reference quantities, with a similarity test being carriedout between a first normalized reference quantity and a secondnormalized reference quantity if the ratio is invariant when beingscaled.